Pronunciation Evaluation in Read and Spontaneous Speech: a Comparison between Human Ratings and Automatic Scores

This paper describes two experiments aimed at exploring the relationship between objective properties of speech and perceived pronunciation quality in read and spontaneous speech, with a view to determining whether such quantitative measures can be used to develop objective pronunciation tests. Read and spontaneous speech of two groups of 60 learners of Dutch as a second language was scored for pronunciation quality by human raters and was analysed by means of a continuous speech recognizer to calculate six quantitative measures of speech quality related to speech timing. The results show that quantitative, temporal measures of speech are strongly related to pronunciation quality, in both read and spontaneous speech, although not all variables suitable for measuring pronunciation quality in read speech are as effective in spontaneous speech. In particular, measures that express the rate at which sounds are produced without taking the frequency and distribution of pauses into account appear to be unsuitable for measuring pronunciation quality in spontaneous speech.